The following methodology, parameters, and R code are based directly on Jenna Vergeynst’s Python code.
Highlighted blocks are quotes from Jenna’s repository.
Use this code to synchronize receivers based on detection times of synchronization transmitters (‘sync-tags’).
The synchronized times can then be processed into a TOA matrix and fed into YAPS to get estimated locations.
Note that the following synchronization process assumes that the sync-tags are embedded within their resepctive receivers (i.e., are not spatially separated).
Synchronization overview
- If possible, apply linear corrections to the receives (e.g., VEMCO’s time drift Auto Correct option in VUE)
- Choose one receiver as the “base receiver”, preferably a receiver in the middle of the array, that hears all of the other receivers. The base receiver is the time keeper, the clock of the other receivers will be synchronised with the base clock.
- Go through the synchronisation process for each receiver. In this process, you compare the detection time of 2 receivers in a receiver pair (base receiver versus receiver in process) detecting the same transmitter (synchronisation transmitter of one of the pair). The goal is to model the spline: a function that reflects the deviation of the receiver clock from the base clock over time.
- Check the figures and play with the parameters of the functions to get a smooth spline that fits (but NOT overfits!) the DTDs as good as possible.
- If you have receivers without synchronisation transmitter, it is still possible to synchronise them:
- Model the spline of the DTD of the base receiver synchronisation transmitter between both receivers
- Convert the distance between both receivers in time-distance by use of soundspeed
- For the resulting spline, substract the time-distance from the base transmitter spline.
Load data
- Detection times from each receiver should be in their own file.
- Supply the timezone for the datetime column
The following sample data for base_rec and other_rec were converted from pickle into csv.
- The pkg_run function will load required libraries (and versions), installing/updating them first as needed
- Read the df’s and configure the datetime column
base_receiver = fread(base_rec_path, sep = ',', header = TRUE)[,
Time := ymd_hms(Time, tz = data_tz)]
other_receiver = fread(other_rec_path, sep = ',', header = TRUE)[,
Time := ymd_hms(Time, tz = data_tz)]
Total rows base_receiver: 2,262,484
Total rows other_receiver: 2,163,807
## Time ID
## 1: 2018-05-01 00:00:03.481616 65001
## 2: 2018-05-01 00:00:06.686088 62209
## 3: 2018-05-01 00:00:08.906914 62215
## 4: 2018-05-01 00:00:10.483454 62214
## 5: 2018-05-01 00:00:13.689360 62211
## 6: 2018-05-01 00:00:13.776442 62059
- Specify the sync ID’s for the base rec and the other rec (bsync_id and osync_id, respectively)
- Specify the datetime and transmitter ID column names in the df’s
DTD and spline of other rec sync-tag
Add DTD
- Subset base_receiver and other_receiver df’s to the detections of the other_rec sync-tag.
- Then, for each detection in the other_receiver sub df, find the nearest detection (within time = t sec) in the base_receiver sub df
- Calculate DTD: the absolute time difference between each detection pair
Chunk execution time: 1 sec
- Plot the DTD
- Adjust plot_subset in order to plot only a subset of the of data, which is useful for large datasets (e.g., plot_subset = 3 plots every 3rd value). Disable this feature by setting to FALSE (or 1).
Smooth DTD
- Smooth the DTD to avoid a subsequent jumpy spline
- Play with the parameters to get the DTD smooth enough (BUT it does not have to be entirely smooth => the spline can handle some noisiness).
- e.g. in the example, if you set window_size=1 with outlier_lim=0.005, you will see the effect of insufficient smoothing
- Zoom in on the figure to check smoothness! (especially at jumps!)
- NOTE: if you smooth less here, the parameter s for modelling the spline will probably have to be larger (in some case this is the best solution).
Chunk execution time: 0.03 sec
Spline DTD
- Split the df at large DTD gaps (> gap_lim sec)
- For each split, model the DTD spline
NOTE: The spline method (and the k_par and s_par paramters) used here (smooth.spline) is different from the one in Jenna’s code (Scipy’s UnivariateSpline), but results should be fairly similar
Chunk execution time: 1.74 sec
- Plot the resulting DTD spline
- Check if there is nice correspondance between spline and DTD
- Zoom in closely to see this well, especially on break points.
- If there is no nice correspondance, play with the parameters of model_spline_part or smooth_DTD.
- For instance:
- if overshoot of the spline at the breaks: you probably need to make the DTD smoother => decrease outlier_lim and/or increase window_size
- if discrepancy between spline and DTD line is too large => decrease window_size (can even be 1)
- if some part is not modelled: probably because it was lost by smoothing => increase outlier_lim
- if the spline is overpredicting (i.e. follows every tiniest variation in DTD_smooth) => increase s and/or decrease k